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Video Messaging 

This application claims the benefit of U.S. Provisional Application No. 60/220,648 
filed July 25, 2000, which is incorporated by reference. 

TECHNICAL FIELD 

The present invention relates generally to transferring electronic data between 
subscribers of a communications system and more particularly to transferring video instant 
messages between subscribers of an instant messaging host. 

BACKGROUND 

Online service providers offer new services and upgrade existing services to enhance 
their subscribers 1 online experience. Subscribers have on-demand access to news, weather, 
financial, sports, and entertainment services as well as the ability to transmit electronic 
messages and to participate in online discussion groups. For example, subscribers of online 
service providers such as America Online or CompuServe may view and retrieve information 
on a wide variety of topics from servers located throughout the world. A server may be 
maintained by the service provider or by a third party provider who makes information and 
services available through the worldwide network of computers that make up the online 
service. 

America Online has provided subscribers with the ability to send and receive instant 
messages. Instant messages are private online conversations between two or more people 
who have subscribed to the instant messaging service and have installed the necessary 
software. Because such online conversations take place virtually in real time, instant 
messaging can provide immediate access to desired information. Instant messaging is 
becoming a preferred means of communicating among online subscribers. 

SUMMARY 

In one general aspect, electronic data is transferred between users of a 
communications system by enabling instant messaging communication between a sender and 
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at least one recipient through an instant messaging host. In addition, video communication is 
enabled between the sender and the recipient through the instant messaging host. 

Implementations may include one or more of the following features. For example, 
implementations may include receiving and authenticating a text instant message from the 
sender at the instant messaging host; determining capabilities of the recipient; reporting the 
capabilities of the recipient; receiving a request to establish video communication from the 
sender and/or the recipient; and/or authenticating the request. Authenticating may include 
identifying a screen name and/or an IP address of the sender and/or the recipient. 
Determining capabilities of the recipient may include identifying hardware or software 
associated with the recipient. A user interface may be displayed according to the capabilities 
of the recipient. 

Video communication maybe enabled by establishing a generic signaling interface 
channel, a control channel, and an audio channel between the sender and the recipient. The 
control channel may include a TCP/IP socket. The audio channel may include a UDP or 
TCP channel. 

These and other general aspects may be implemented by an apparatus and/or by a 
computer program stored on a computer readable medium. The computer readable medium 
may comprise a disc, a client device, a host device, and/or a propagated signal. 

Other features and advantages will be apparent from the following description, 
including the drawings, and from the claims. 

DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a block diagram of a communications system. 

Figs. 2-5 are block diagrams of expansions of aspects of the communications system 
of Fig. 1. 

Fig. 6 is a flow chart of a communications method that may be implemented by the 
communications systems of Figs. 1-5. 

Figs. 7-13 are illustrations of different graphical user interfaces that may be presented 
by the communications systems of Figs 1-5. 
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DETAILED DESCRIPTION 

For illustrative purposes, Figs. 1-5 show aspects of a communications system for 
transferring electronic data between a client and a host. For brevity, several elements in the 
figures are represented as monolithic entities. Such elements, however, may represent 
5 numerous interconnected computer systems and/or components. An addressing scheme such 
as, for example, Uniform Resource Locators ("URLs") may be used to define the location 
and type of each element and/or component of the communications system. 

The terms "client" and "host" generally refer to a role as a requester of data (client) or 
a provider of data (host). For example, data requested by a client may be transferred directly 
1 0 or indirectly from a host through a network and, finally, to the client. Elements of the 

communications system, however, may request data in one transaction and provide data in 
another transaction, thus having dual and/or changing roles. 

3 Referring to Fig. 1, a communications system 100 is capable of delivering and 
p exchanging data between a client system 105 and a host system 110 through a 

75 communications link 115. The client system 105 may include one or more client devices 120 

4 and/or client controllers 125, and the host system 110 may include one or more host devices 

5 135 and/or host controllers 140. For example, the client system 105 or the host system 110 
may include one or more general-purpose computers (e.g., personal computers), one or more 

4 special-purpose computers (e.g., devices specifically programmed to communicate with each 
|o other and/or the client system 1 05 or the host system 1 1 0), or a combination of one or more 
3 general-purpose computers and one or more special-purpose computers. The client system 

■is 

105 and the host system 110 may be arranged to operate within or in concert with one or 
more other systems, such as, for example, one or more local area networks ("LANs") and/or 
one or more wide area networks ("WANs"). The client device 120, the client controller 125, 

25 the host device 135, and the host controller 140 each may include one or more hardware 
components and/or software components. 

In general, a device (e.g., client device 120 and/or host device 135) executes 
instructions under the command of a controller (e.g., client controller 125 and/or host 
controller 140) and is connected to such controller by a wired and/or wireless data pathway 

30 (e.g., pathway 130 and/or pathway 145) capable of delivering data. An example of a device 
(e.g., client device 120 and/or host device 135) is a general-purpose computer (e.g., a 
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personal computer, server) capable of responding to and executing instructions in a defined 
manner. Other examples include a special-purpose computer, a workstation, a component, 
other physical or virtual equipment and/or some combination thereof capable of responding 
to and executing instructions and/or capable of peer-to-peer communications. 
5 An example of a controller (e.g., client controller 125 and/or host controller 140) is a 

software application for commanding and directing communications. Other examples 
include a program, a piece of code, an instruction, a computer, a computer system, and/or a 
combination thereof, for independently or collectively instructing a device (e.g., client device 
120 and/or host device 135) to interact and operate as described. A controller (e.g., client 
1 0 controller 125 and/or host controller 140) may be embodied permanently or temporarily in 
any type of machine, component, physical or virtual equipment, storage medium, or 
propagated signal capable of providing instructions to a device (e.g., client device and/or host 
O device 135). 

Jrj The communications link 115 may include a delivery network 160 for directly or 

H5 indirectly connecting the client system 105 and the host system 1 10, irrespective of physical 
vj- separation. The delivery network 160 may include one or more interconnected networks 
;S such as, for example, the Internet, the World Wide Web ("Web"), a WAN, a LAN, an analog 

* m and/or a digital wired or wireless telephone network (e.g., PSTN, ISDN, and xDSL), a radio 

O 

Kj, network, a television network, a cable network, a satellite network, and/ or any other delivery 
!|o mechanism for carrying data. The delivery network 160 also may include several 
O intermediate and/or routing devices, such as, for example, proxy servers, bridges, and routers. 
The communications link 115 may include one or more communication pathways (e.g., 
pathway 150 and/or pathway 155) that enable communications through the delivery network 
160. Each communication pathway (e.g., pathway 150 and/or pathway 155) may include, for 
25 example, a wired, wireless, cable or satellite communication pathway. 

Fig. 2 illustrates a communications system 200 including a client system 205 
communicating with a host system 210 through a communications link 215. The client 
system 205 may include one or more client devices 220 and one or more client controllers 
225 for controlling the client devices 220. The host system 210 may include one or more 
30 host devices 235 and one or more host controllers 240 for controlling the host devices 235. 
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The communications link 215 may include communication pathways 250, 255 enabling 
communications through the one or more delivery networks 260. 

Examples of each element within the communications system of Fig. 2 are broadly 
described above with respect to Fig. 1 . In particular, the host system 210 and 
5 communications link 215 may have attributes comparable to those described with respect to 
host system 110 and communications link 115 of Fig. 1. Likewise, the client system 205 of 
Fig. 2 may have attributes comparable to and illustrates one possible implementation of the 
client system 105 of Fig. 1. 

In one implementation, the client device 220 includes a general-purpose computer 
10 270 having an internal or external storage 272 for storing data and programs such as an 

operating system 274 (e.g., DOS, Windows™, Windows 95™, Windows 98™, Windows 
2000™, Windows Me™, Windows XP™, Windows NT™, OS/2, or Linux) and one or more 
[3 application programs. Examples of application programs include authoring applications 276 
Jg (e.g., word processing, database programs, spreadsheet programs, or graphics programs) 
ff 5 capable of generating documents or other electronic content; client applications 278 (e.g., 
H AOL client, CompuServe client, AIM client, AOL TV client, or ISP client) capable of 
communicating with other computer users, accessing various computer resources, and 
viewing, creating, or otherwise manipulating electronic content; and browser applications 

ill 

-jj 280 (e.g., Netscape's Navigator, Microsoft's Internet Explorer, Java's microbrowser) capable 
= |o of rendering Internet content. 

p The general-purpose computer 270 also includes a central processing unit 282 

("CPU") for executing instructions in response to commands from the client controller 225. 
The client controller 225 may include one or more of the application programs installed on 
the internal or external storage 272 of the general-purpose computer 270. The client 

25 controller 225 also may include application programs externally stored in and performed by 
one or more device(s) external to the general- purpose computer 270. 

The general-purpose computer 270 includes a communication device 284 for sending 
and receiving data. One example of the communication device 284 is a modem. Other 
examples include a transceiver, a set-top box, a communication card, a satellite dish, an 

30 antenna, or another network adapter capable of transmitting and receiving data over the 
communications link 215 through a wired or wireless data pathway 250. The general- 
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purpose computer 270 also may include a television ('TV") tuner 286 for receiving TV 
programming in the form of a broadcast reception, satellite transmission, and/or cable signal. 
As a result, the client device 220 can selectively and/or simultaneously display network 
content received by communications device 284 and television programming content 
5 received by the TV tuner 286. 

The general-purpose computer 270 also includes an input/output interface 288 for 
wired or wireless connection to various peripheral devices 290. Examples of peripheral 
devices 290 include, but are not limited to, a mouse 291, a mobile phone 292, a personal 
digital assistant 293 ("PDA"), an MP3 player (not shown), a keyboard 294, a display monitor 
10 295 with or without a touch screen input, a TV remote control 296 for receiving information 
from and rendering information to subscribers, and an audiovisual input device 298 (e.g., 
Web cam, video camera, microphone, speakers). 
Q Although Fig. 2 illustrates devices such as a mobile telephone 292, a PDA 293, an 

;jj MP3 player (not shown), and a TV remote control 296 as being peripheral with respect to the 
Hs general-purpose computer 270, in another implementation, such devices may themselves 
*%j include the functionality of the general-purpose computer 270 and operate as the client 
^ device 220. For example, the mobile phone 292 or the PDA 293 may include computing and 
^ networking capabilities and function as a client device 220 by accessing the delivery network 
%l 260 and communicating with the host system 210. Furthermore, the client system 205 may 
jj i) include one, some or all of the components and devices described above. 
C3 Referring to Fig. 3, a communications system 300 is capable of delivering and 

exchanging information between a client system 305 and a host system 3 10 through a 
communication link 3 1 5. The client system 305 may include one or more client devices 320 
and one or more client controllers 325 for controlling the client devices 320. The host 
25 system 3 1 0 may include one or more host devices 335 and one or more host controllers 340 
for controlling the host devices 335. The communications link 315 may include 
communication pathways 350, 355 enabling communications through the one or more 
delivery networks 360. 

Examples of each element within the communications system of Fig. 3 are broadly 
30 described above with respect to Figs. 1 and 2. In particular, the client system 305 and the 
communications link 315 may have attributes comparable to those described with respect to 
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client systems 105 and 205 and communications links 115 and 215 of Figs. 1 and 2. 
Likewise, the host system 310 of Fig. 3 may have attributes comparable to and illustrates one 
possible implementation of the host systems 110 and 210 shown in Figs. 1 and 2. 

The host system 310 includes a host device 335 and a host controller 340. The host 
5 controller 340 is generally capable of transmitting instructions to any or all of the elements of 
the host device 335. For example, in one implementation, the host controller 340 includes 
one or more software applications loaded on the host device 335. In other implementations, 
as described above, the host controller 340 may include any of several other programs, 
machines, and devices operating independently or collectively to control the host device 335. 
10 The host device 335 includes a login server 370 for enabling access by subscribers 

and for routing communications between the client system 305 and other elements of the host 
device 335. The host device 335 also includes various host complexes such as the depicted 
O OSP ("Online Service Provider") host complex 380 and IM ("Instant Messaging") host 
v g complex 390. To enable access to these host complexes by subscribers, the client system 305 
l% includes communication software, for example, an OSP client application and an IM client 
H application. The OSP and IM communication software applications are designed to facilitate 
*g the subscriber's interactions with the respective services and, in particular, may provide 

access to all the services available within the respective host complexes. 
H Typically, the OSP host complex 380 supports different services, such as email, 

discussion groups, chat, news services, and Internet access. The OSP host complex 380 is 
O generally designed with an architecture that enables the machines within the OSP host 

complex 380 to communicate with each other and employs certain protocols (i.e., standards, 
formats, conventions, rules, and structures) to transfer data. Examples of protocols include, 
but are not limited to hypertext transfer protocol ("HTTP"), user datagram protocol ("UDP"), 
25 and/or layer two tunneling protocol ("L2TP"). The OSP host complex 380 also may employ 
one or more proprietary OSP protocols and custom dialing engines to enable access by 
selected client applications. The OSP host complex 380 may define one or more specific 
protocols for each service based on an underlying protocol. 

The IM host complex 390 may be independent of or included in the OSP host 
30 complex 380. The IM host complex 390 may support instant messaging services for OSP 

subscribers as well as for subscribers to other networks. Thus, the IM host complex 390 may 
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enable instant messaging irrespective of an IM subscriber's Internet service provider. The IM 
host complex 390 also may support associated services, such as administrative support, 
advertising, directory services, chat, and interest groups related to the instant messaging. The 
IM host complex 390 has an architecture that enables all of the machines within the IM host 
5 complex 390 to communicate with each other. The IM host complex may employ one or 
more standard and/or proprietary protocols to transfer data. 

The host device 335 may include one or more gateways (e.g., OSP host complex 
gateway 385 and/or IM host complex gateway 395) that connect and link complexes (e.g., 
the OSP host complex 380 and the IM host complex 390. Such gateways may directly or 
1 0 indirectly link host complexes through wired and/or wireless pathways. Ordinarily, when 

used to facilitate a link between complexes, a gateway (e.g., OSP host complex gateway 385 
and/or IM host complex gateway 395) is privy to information regarding the protocol type 
O anticipated by a destination complex, which enables any necessary protocol conversion to be 

performed incident to the transfer of data from one complex to another. For instance, the 
[£5 OSP host complex 380 and IM host complex 390 may use different protocols such that 
s 4 transferring data between the complexes requires protocol conversion by or at the request of 
, jj a gateway (e.g., OSP host complex gateway 385 and/or the IM host complex gateway 395). 
JL Referring to Fig. 4, a communications system 400 is capable of delivering and 

H exchanging information between a client system 405 and a host system 410 through a 
jlo communication link 415. The client system 405 may include one or more client devices 420 
^ and one or more client controllers 425 for controlling the client devices 420. The host 

system 410 may include one or more host devices 435 and one or more host controllers 440 
for controlling the host devices 435. The communications link 415 may include 
communication pathways 450, 455 enabling communications through the one or more 
25 delivery networks 460. As shown, the client system 405 may access the Internet 465 through 
the host system 410. 

Examples of each element within the communications system of Fig. 4 are broadly 
described above with respect to Figs. 1-3. In particular, the client system 405 and the 
communications link 415 may have attributes comparable to those described with respect to 
30 client systems 105, 205, and 305 and communications links 115, 215, and 315 of Figs. 1-3. 

Likewise, the host system 410 of Fig. 4 may have attributes comparable to and illustrates one 
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possible implementation of the host systems 110, 210, and 310 shown in Figs. 1-3. Fig. 4 
describes an aspect of the host system 410, focusing primarily on one particular 
implementation of OSP host complex 480. 

The client system 405 includes a client device 420 and a client controller 425, The 
5 client controller 425 is generally capable of establishing a connection to the host system 410, 
including the OSP host complex 480, the IM host complex 490 and/or the Internet 465. In 
one implementation, the client controller 425 includes an OSP application for communicating 
with servers in the OSP host complex 480 using exclusive OSP protocols. The client 
controller 425 also may include applications, such as an IM client application, and/or an 
1 0 Internet browser application, for communicating with the IM host complex 490 and the 
Internet 465. 

The host system 410 includes a host device 435 and a host controller 440. In general, 
O the host controller 440 is capable of transmitting instructions to any or all of the elements of 
\y the host device 435. For example, in one implementation, the host controller 440 includes 
[?5 one or more software applications loaded on one or more elements of the host device 435. In 
N other implementations, as described above, the host controller 440 may include any of 
several other programs, machines, and devices operating independently or collectively to 
control the host device 435. 
H The host system 410 includes a login server 470 capable of enabling communications 

l|o with and authorizing access by client systems 405 to various elements of the host system 410, 
including an OSP host complex 480 and an IM host complex 490. The login server 470 may 
implement one or more authorization procedures to enable simultaneous access to the OSP 
host complex 480 and the IM host complex 490. The OSP host complex 480 and the IM host 
complex 490 are connected through one or more OSP host complex gateways 485 and one or 
25 more IM host complex gateways 495. Each OSP host complex gateway 485 and IM host 
complex gateway 495 may perform any protocol conversions necessary to enable 
communications between the OSP host complex 480, the IM host complex 490, and the 
Internet 465. 

The OSP host complex 480 supports a set of services from one or more servers 
30 located internal to and external from the OSP host complex 480. Servers external to the OSP 
host complex 480 generally may be viewed as existing on the Internet 465. Servers internal 
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to the OSP complex 480 may be arranged in one or more configurations. For example, 
servers may be arranged in centralized or localized clusters in order to distribute servers and 
subscribers within the OSP host complex 480. 

In one implementation of Fig. 4, the OSP host complex 480 includes a routing 
5 processor 4802. In general, the routing processor 4802 will examine an address field of a 
data request, use a mapping table to determine the appropriate destination for the data 
request, and direct the data request to the appropriate destination. In a packet-based 
implementation, the client system 405 may generate information requests, convert the 
requests into data packets, sequence the data packets, perform error checking and other 
1 0 packet-switching techniques, and transmit the data packets to the routing processor 4802. 
Upon receiving data packets from the client system 405, the routing processor 4802 may 
directly or indirectly route the data packets to a specified destination within or outside of the 
O OSP host complex 480. For example, in the event that a data request from the client system 
2 405 can be satisfied locally, the routing processor 4802 may direct the data request to a local 
Hs server 4804. In the event that the data request cannot be satisfied locally, the routing 
\j processor 4802 may direct the data request externally to the Internet 465 or the IM host 
^ complex 490 through the gateway 485. 

^ The OSP host complex 480 also includes a proxy server 4806 for directing data 

^ requests and/or otherwise facilitating communication between the client system 405 and the 
! |o Internet 465. The proxy server 4806 may include an IP ("Internet Protocol") tunnel for 
□ converting data from OSP protocol into standard Internet protocol and transmitting the data 
to the Internet 465. The IP tunnel also converts data received from the Internet 465 in the 
standard Internet protocol back into the OSP protocol and sends the converted data to the 
routing processor 4802 for delivery back to the client system 405. 
25 The proxy server 4806 also may allow the client system 405 to use standard Internet 

protocols and formatting to access the OSP host complex 480 and the Internet 465. For 
example, the subscriber may use an OSP TV client application having an embedded browser 
application installed on the client system 405 to generate a request in standard Internet 
protocol, such as HTTP. In a packet-based implementation, data packets may be 
30 encapsulated inside a standard Internet tunneling protocol, such as UDP, for example, and 
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routed to the proxy server 4806. The proxy server 4806 also may include an L2TP tunnel 
capable of establishing a point-to-point protocol ("PPP") session with the client system 405. 

The proxy server 4806 may act as a buffer between the client system 405 and the 
Internet 465, and may implement content filtering and time saving techniques. For example, 
the proxy server 4806 can check parental controls settings of the client system 405 and 
request and transmit content from the Internet 465 according to the parental control settings. 
In addition, the proxy server 4806 may include one or more caches for storing frequently 
accessed information. If requested data is determined to be stored in the caches, the proxy 
server 4806 may send the information to the client system 405 from the caches and avoid the 
need to access the Internet 465. 

Referring to Fig. 5, a communications system 500 is capable of delivering and 
exchanging information between a client system 505 and a host system 510 through a 
communication link 515. The client system 505 may include one or more client devices 520 
and one or more client controllers 525 for controlling the client devices 520. The host 
system 510 may include one or more host devices 535 and one or more host controllers 540 
for controlling the host devices 535. The communications link 515 may include 
communication pathways 550, 555 enabling communications through the one or more 
delivery networks 560. As shown, the client system 505 may access the Internet 565 through 
the host system 510. 

Examples of each element within the communications system of Fig. 5 are broadly 
described above with respect to Figs. 1-4. In particular, the client system 505 and the 
communications link 515 may have attributes comparable to those described with respect to 
client systems 105, 205, 305, and 405 and communications links 115, 215, 315, and 415 of 
Figs. 1-4. Likewise, the host system 510 of Fig. 5 may have attributes comparable to and 
illustrates one possible implementation of the host systems 1 10, 210, 310, and 410 shown in 
Figs. 1-4. Fig. 5 describes an aspect of the host system 510, focusing primarily on one 
particular implementation of the DM host complex 590. 

The client system 505 includes a client device 520 and a client controller 525. In 
general, the client controller 525 is capable of establishing a connection to the host system 
510, including the OSP host complex 580, the IM host complex 590 and/or the Internet 565. 
In one implementation, the client controller 525 includes an IM application for 
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communicating with servers in the IM host complex 590 utilizing exclusive (i.e., proprietary) 
IM protocols. The client controller 525 also may include applications, such as an OSP client 
application, and/or an Internet browser application for communicating with the OSP host 
complex 580 and the Internet 565, respectively. 
5 The host system 510 includes a host device 535 and a host controller 540. In general, 

the host controller 540 is capable of transmitting instructions to any or all of the elements of 
the host device 535. For example, in one implementation, the host controller 540 includes 
one or more software applications loaded on one or more elements of the host device 535. 
However, in other implementations, as described above, the host controller 540 may include 
10 any of several other programs, machines, and devices operating independently or collectively 
to control the host device 535. 

The host system 510 includes a login server 570 capable of enabling communications 
Q with and authorizing access by client systems 505 to various elements of the host system 510, 
ai including an OSP host complex 580 and an IM host complex 590. The login server 570 may 
f Is implement one or more authorization procedures to enable simultaneous access to the OSP 
% l host complex 580 and the IM host complex 590. The OSP host complex 580 and the IM host 
~fl complex 590 are connected through one or more OSP host complex gateways 585 and one or 
JL more IM host complex gateways 595. Each OSP host complex gateway 585 and IM host 
H complex gateway 595 may perform any necessary protocol conversions to enable 
j §0 communication among the OSP host complex 580, the IM host complex 590, and/or the 
^ Internet 565. 

In one implementation, to begin an instant messaging session, the client system 505 
accesses the IM host complex 590 and establishes a connection to the login server 570. The 
login server 570 determines whether the particular subscriber is authorized to access the IM 
25 host complex 590 by verifying a subscriber identification and password. If the subscriber is 
authorized to access the IM host complex 590, the login server 570 employs a hashing 
technique on the subscriber's screen name to identify a particular IM server 5902 for use 
during the subscriber's session. The login server 570 provides the client system 505 with the 
IP address of the particular IM server 5902, gives the client system 505 an encrypted key 
30 (i.e., a cookie), and breaks the connection. The client system 505 then uses the IP address to 
establish a connection to the particular IM server 5902 through the communications link 515, 
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and obtains access to that IM server 5902 using the encrypted key. The client system 505 
may be equipped with a Winsock application programming interface ("API") that enables the 
client system 505 to establish an open transmission control protocol ("TCP") connection to 
the IM server 5902. 

Once a connection to the IM server 5902 has been established, the client system 505 
may directly or indirectly transmit data to and access content from the IM server 5902 and 
one or more associated domain servers 5904. The IM server 5902 supports the fundamental 
instant messaging services and the domain servers 5904 may support associated services, 
such as, for example, administrative matters, directory services, chat and interest groups. In 
general, the purpose of the domain servers 5904 is to lighten the load placed on the IM server 
5902 by assuming responsibility for some of the services within the IM host complex 590. 
By accessing the IM server 5902 and/or the domain server 5904, a subscriber can use the IM 
client application to view whether particular subscribers ("buddies") are online, exchange 
instant messages with particular subscribers, participate in group chat rooms, trade files such 
as pictures, invitations or documents, find other subscribers with similar interests, get 
customized news and stock quotes, and search the World Wide Web. 

In the implementation of Fig. 5, the IM server 5902 is directly or indirectly connected 
to a routing gateway 5906. The routing gateway 5906 facilitates the connection between the 
IM server 5902 and one or more alert MUXs ("MUXs") 5908, for example, by serving as a 
link minimization tool or hub to connect several IM servers 5902 to several alert MUXs 
5908. In general, an alert MUX 5908 maintains a record of alerts and subscribers registered 
to receive the alerts. 

Once the client system 505 is connected to the alert MUX 5908, a subscriber can 
register for and/or receive one or more types of alerts. The connection pathway between the 
client system 505 and the alert MUX 5908 is determined by employing another hashing 
technique at the IM server 5902 to identify the particular alert MUX 5908 to be used for the 
subscriber's session. Once the particular MUX 5908 has been identified, the IM server 5902 
provides the client system 505 with the IP address of the particular alert MUX 5908 and 
gives the client system 505 an encrypted key (i.e., a cookie). The client system 505 then uses 
the IP address to connect to the particular alert MUX 5908 through the communication link 
515 and obtains access to the alert MUX 5908 using the encrypted key. 
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The alert MUX 5908 is connected to an alert gate 5910 that, like the IM host complex 
gateway 595 , is capable of performing the necessary protocol conversions to form a bridge to 
the OSP host complex 580. The alert gate 5910 is the interface between the IM host complex 
590 and the physical servers, such as servers in the OSP host complex 580, where state 
5 changes are occurring. In general, the information regarding state changes will be gathered 
and used by the IM host complex 590. However, the alert MUX 5908 also may 
communicate with the OSP host complex 580 through the IM host complex gateway 595, for 
example, to provide the servers and subscribers of the OSP host complex 580 with certain 
information gathered from the alert gate 5910. 
10 The alert gate 5910 can detect an alert feed corresponding to a particular type of alert. 

The alert gate 5910 may include a piece of code (i.e., alert receive code) capable of 
interacting with another piece of code (i.e., alert broadcast code) on the physical server where 
U a state change occurs. In general, the alert receive code installed on the alert gate 5910 

instructs the alert broadcast code installed on the physical server to send a feed to the alert 
f 15 gate 5910 upon the occurrence of a particular state change. Upon detecting a feed, the alert 
% i gate 5910 contacts the alert MUX 5908, which in turn, informs the client system 505 of the 
detected feed. 

JL In the implementation of Fig. 5, the IM host complex 590 also includes a subscriber 

H profile server 5912 connected to a database 5914 for storing large amounts of subscriber 
j §0 profile data. The subscriber profile server 5912 may be used to enter, retrieve, edit, 
} J manipulate, or otherwise process subscriber profile data. In one implementation, a 

subscriber's profile data includes, for example, the subscriber's buddy list, alert preferences, 
designated stocks, identified interests, and geographic location. The subscriber may enter, 
edit and/or delete profile data using an installed IM client application on the client system 
25 505 to interact with the subscriber profile server 5912. 

Because the subscriber's data is stored in the IM host complex 590, the subscriber 
does not have to reenter or update such information in the event that the subscriber accesses 
the IM host complex 590 using a new and/or different client system 505. Accordingly, when 
a subscriber accesses the IM host complex 590, the IM server 5902 can instruct the 
30 subscriber profile server 5912 to retrieve the subscriber's profile data from the database 5914 
and to provide, for example, the subscriber's buddy list to the IM server 5902 and the 
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subscriber's alert preferences to the alert MUX 5908. The subscriber profile server 5912 also 
may communicate with other servers in the OSP host complex 580 to share subscriber profile 
data with other services. Alternatively, user profile data may be saved locally on the client 
device 505. 

Referring to Fig. 6, a sender 602a, a recipient 602b, and a host 604 transfer video data 
according to a procedure 600. The procedure 600 may be implemented by any suitable type 
of hardware (e.g., device, computer, computer system, equipment, component); software 
(e.g., program, application, instructions, code); storage medium (e.g., disk, external memory, 
internal memory, propagated signal); or combination thereof. 

Examples of each element of Fig. 6 are broadly described with respect to Figs. 1-5 
above. In particular, the sender 602a and the recipient 602b may have attributes comparable 
to those described with respect to client devices 120, 220, 320, 420, and 520 and/or client 
controllers 125, 225, 325, 425, and 525. The host 604 may have attributes comparable to 
those described with respect to host device 135, 235, 335, 435, and 535 and/or host 
controllers 140, 240, 340, 440, and 540. The sender 602a, the recipient 602b, and/or the host 
604 maybe directly or indirectly interconnected through a known or described delivery 
network. 

In one implementation, the sender 602a is associated with a first subscriber and the 
recipient 602b is associated with a second subscriber, and each of the sender 602a and the 
recipient 602b include a client application for accessing the host 604. Each subscriber may 
use the client application to set individual preferences for allowing messages and/or files to 
be transferred to and from other subscribers. Typically, a graphical user interface ("UI") is 
displayed that allows each subscriber to select among various levels of security and/or to 
grant (or deny) access to others subscribers. A subscriber's transfer preferences maybe 
maintained locally at the client or remotely at the host 604. In this example, the transfer 
preferences are set to allow messages and files to be transferred between the sender 602a and 
the recipient 602b. 

In order to communicate using instant messaging, the sender 602a and the recipient 
602b must be accessing the host 604 at the same time. To access the host 604, the sender 
602a and the recipient 602b each send a separate request to the host 604. The request 
identifies the associated subscriber to the host 604 and to other subscribers using a unique 
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screen name. The host 604 verifies a subscriber's information (e.g., screen name and 
password) against data stored in a subscriber database. If the subscriber's information is 
verified, the host 604 authorizes access. If the subscriber's information is not verified, the 
host 604 denies access and sends an error message. 
5 After being authorized, the sender 602a and the recipient 602b can communicate over 

a direct (i.e., socket) connection established through the host 604. The sender 602a and the 
recipient 602b use the connection to communicate with the host 604 and with each other. 
The connection remains open during the time that the sender 602a and the recipient 602b are 
accessing the host 604. The sender 602a and the recipient 602b each may include a Winsock 
10 API for opening and establishing a TCP connection to the host 604. 

Upon accessing the host 604, a "buddy list" is displayed to the subscriber. In general, 
a subscriber's buddy list is a user interface that lists the online status and capabilities of 

0 certain screen names, i.e., "buddies," identified by the subscriber. In particular, the host 604 
,g informs the sender whether identified buddies are online, i.e., currently accessing the host 

f fs 604. The host 604 also informs any subscriber who has identified the sender as a buddy that 

H the sender is currently online. The buddy list also facilitates instant messaging 

s f | communication between subscribers. A subscriber can activate an instant messaging 

message user interface pre-addressed to a buddy simply by clicking the screen name of a 
y buddy on the buddy list. If a recipient is not a "buddy," the first subscriber must activate a 

1 ?fo blank instant messaging user interface and then address it to the screen name of the intended 
O recipient. When necessary, a subscriber can look up the screen name of an intended recipient 

using the intended recipient's e-mail address. In addition to exchanging instant messages 
with online buddies, the sender may participate in group chat rooms, locate other subscribers 
with similar interests, get customized news and stock quotes, search the Web, and transfer 

25 files to and from other subscribers. 

Video messaging (i.e., video-enabled instant messaging) further extends the 
functionality of instant messaging by allowing the sender 602a and the recipient 602b to 
communicate peer to peer using video, i.e., camera, microphone, and speaker. In the 
implementation of Fig. 6, a sender 602a, a recipient 602b, and a host 604 interact according 

30 to a procedure 600 to send and receive a video message (i.e., video instant message). 
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Initially, the sender 602a accesses the host 604 (step 606). In one implementation, 
the sender 602a is a subscriber and/or a client (e.g., client system 505) and the host 604 
includes one or more host complexes (e.g., OSP host complex 580 and/or IM host complex 
590) for providing instant messaging capability and coordinating the transfer of electronic 
5 data between subscribers. The sender 602a may access the host 604 using any available 

device (e.g., computer, PC, laptop, appliance, pager, PDA, interactive TV, telephone) and/or 
a controller (e.g., software program, client application, browser application). 

The sender 602a designates at least one recipient 602b to receive an instant message 
(step 608), generates an instant message (step 610) and then transmits the instant message to 
10 the host 604 (step 612). The instant message may be, for example, a text instant message or 
other non- video instant message (e.g., voice message) created by the sender 602a. The 
recipient 602b may be a subscriber and/or client (e.g., client system 505) capable of 
y accessing the host 604 using a device (e.g., computer, PC, laptop, appliance, pager, PDA, 
v3 interactive TV, telephone) and/or a controller (e.g., software program, client application, 
f 15 browser application). In one implementation, a screen name associated with the intended 
H recipient 602b has been identified as a "buddy" of the sender 602a, and a UI (e.g., buddy list) 

indicating the online status and capabilities of the recipient 602b is displayed to the sender 
JL 602a. Thus, the sender 602a can confirm that the recipient is able to communicate (i.e., is 
H! online) and then open an IM box by selecting (e.g., clicking) the screen name associated with 
j SO the recipient 602b. After composing an instant message and clicking a Send button, the 
sender 602a transmits the instant message to the host 604 (step 612). 

The host 604 receives the instant message from the sender 602a (step 614) and then 
authenticates the instant message (step 616). In one implementation, the instant message 
includes header information identifying the message type, the screen name and/or IP address 
25 of the sender 602a and the recipient 602b, and a randomly generated security number. A 
server (e.g., IM server 5902) on the host 604 may authenticate the instant message by 
matching the screen names and/or IP addresses with those of valid subscribers stored in a 
reverse look-up table. In the event that either the sender 602a or recipient 602b is not 
associated with a valid subscriber, the host 604 reports an error message. 
30 After authenticating the instant message (step 616), the host 604 detects the 

capabilities of the recipient (step 618) and reports the capabilities of the recipient 602b to the 
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sender 602a (step 620). In one implementation, a network of servers (e.g., EM servers 5902) 
on the host 604 monitors and updates the online status, client version, and device type of 
connected subscribers in real time. The capability to use video messaging (i.e., to receive a 
video instant message) may depend on factors such as a subscriber's hardware (e.g., device 
5 type), software (e.g., client version), and/or transfer preferences (e.g., blocked screen names). 
In general, to be video-enabled, the necessary software (e.g., video-enabled IM client 
application) and video equipment (e.g., audiovisual input device 298) must be available to 
the sender 602a and the recipient 602b. 

Next, the sender 602a receives the report from the host 604 (step 622) and displays a 
10 UI corresponding to the capabilities of the sender 602a and/or the recipient 602b (step 624). 
In general, if the sender 602a (e.g., client system 505) is not video-enabled, the sender 602a 
displays a standard instant messaging UI. If the sender 602a is video-enabled, then the 
Q sender 602a displays a video-enabled UI. In one implementation, shown in Fig. 7, a Video 
in Messaging Buddy List UI 700 includes a buddy list 702 and a Video Message button 704 for 
f?5 instructing the host 604 to request a video message connection when clicked. The UI 700 
HI also may include other buttons (not shown) corresponding to additional capabilities (e.g., 

voice M) of the sender 602a and/or the recipient 602b. 
:L hi another implementation, shown in Fig. 8, a Start Video Message UI 800 includes 

^4 an IM window 805 for displaying a running transcript of an IM session, a text message area 
| |o 810 for entering the text of an instant message, and an IM toolbar 812 for changing text or 
O background colors, changing text size, emphasizing text (e.g., bold, italic, or underlining), 
and inserting objects (e.g., emoticons, hyperlinks, images). The Start Video Message UI 800 
also includes IM buttons 814 for performing IM functions such as sending instant messages, 
canceling an instant message or IM session, getting the profile of a sender, and notifying the 
25 OSP of offending conduct. In addition, the Start Video Message UI 800 includes a 

notification 816 and a Start VM (Video Message) button 818 for instructing the host 604 to 
request a video message connection when clicked. 

After reporting the capabilities of the recipient 602b to the sender 602a (step 620) or 
at any time after authenticating the instant message (step 616), the host 604 sends the instant 
30 message to the recipient 602b (step 626). The recipient 602b accepts the instant message 

from the host 604 (step 628) and displays a UI corresponding to the capabilities of the sender 
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602a and/or the recipient 602b (step 630). In general, if the recipient 602b is not video- 
enabled, then the recipient 602b displays a standard instant messaging UI. If the recipient 
602b is video-enabled, the recipient 602b displays a video messaging UI (e.g., UI 700, UI 
800). 

5 In the event both the sender 602a and the recipient 602b are video-enabled, either can 

initiate a video message session. In the implementation of Fig. 6, the sender 602a initiates a 
video message session by sending a connect request to the host 604 (step 632). The connect 
request may contain information including, but not limited to, the message type, the screen 
name and/or IP address of the sender 602a and recipient 602b, and a randomly generated 
10 security number. The connect request may be created and sent automatically by clicking the 
Video Message button 710 of the UI 700 and/or the Start VM button 818 of the UI 800, for 
example. 

0 The host 604 receives the connect request from the sender 602a (step 634), 

, g authenticates the connect request from the sender 602a (step 636), and then sends the connect 
ff5 request to the recipient 602b (step 638). The host 604 may authenticate the video request, for 
N example, by using a reverse look-up table to match the screen names and/or IP addresses 

with those of valid subscribers. In the event that either the sender 602a or recipient 602b is 
^ not associated with a valid subscriber, the host 604 reports an error message. 
%i The recipient 602b receives the connect request (step 640) and then displays a UI 

1 So informing the recipient 602b that the sender 602a wants to engage in a video message session 
O (step 642). In one implementation, shown in Fig. 9, a Connect UI 900 includes an M 

window 905 for displaying a running transcript of an IM session, a text message area 910 for 
entering the text of an instant message, and an IM toolbar 912 for changing text or 
background colors, changing text size, emphasizing text (e.g., bold, italic, or underlining), 

25 and inserting objects (e.g., emoticons, hyperlinks, images). The Connect UI 900 also 

includes IM buttons 914 for performing IM functions such as sending instant messages, 
canceling an instant message or IM session, getting the profile of a sender, and notifying the 
OSP of offending conduct. In addition, the Connect UI 900 includes a notification 916 and a 
connect button 918 for authorizing the host 604 to establish a video message connection 

30 when clicked. 
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When presented with the connect request, the recipient 602b may ignore the connect 
request, accept the connect request, or cancel the instant message session. If the recipient 
602b accepts the connect request (step 644), for example, by clicking the connect button 818, 
the host 604 establishes a video message session (step 646) that allows the sender 602a to 
5 communicate with the recipient 602b using video messaging. 

After the host 604 establishes a video message session (step 646), the sender 602a 
generates a video message (step 648). In one implementation, shown in Fig. 10A, the sender 
602a is presented with an Instant Video Message UI 1000. The Instant Video Message UI 
1000 includes an IM window 1005 for displaying a running transcript of an IM session and a 
10 text message area 1010 for entering the text of an instant message. The Instant Video 

Message UI 100 also includes a send video message window 1015 for recording, previewing, 
n and/or sending a video message, a transfer indicator 1020 for indicating the transfer of data 
%0 from the sender 602a, a Record button 1025 to start recording a video message, a Send button 
La 1030 to send a recorded video message, and Clear button 1035 to delete a recorded video 
lip message. A screen name associated with the sender 602a is displayed at the top of the send 
tfl video message window 1015. The Instant Video Message UI 1000 additionally includes a 
g"" receive move message window 1040 for viewing a received video message and a transfer 

indicator 1045 for indicating the transfer of data to the sender 602b. A screen name 
ffj- associated with the recipient 602b is displayed at the top of the receive video message 
fgjo window 1040. 

^ Referring to Fig. 10B, a subscriber associated with the sender 602a (e.g., computer 

system running a video-enabled instant messaging client) may generate a video message 
(step 648) by clicking the Record button 1025 and speaking a message into video recording 
equipment such as, for example, a video camera with a speaker or other recording device 

25 (e.g., audiovisual device 298) capable of capturing sight and sound. In one implementation, 
there are recording restrictions (e.g., time, size, number) for video messages placed on the 
sender 602a. The sender 602a (e.g., IM client application) and/or the host 604 (e.g., IM 
5902) may impose and/or enforce the recording restrictions. For example, the sender may 
not be able to record a video message longer than 15 seconds. The client 602a may be 

30 configured, however, to automatically send a video message when the limit is reached and to 
begin recording another video message without user intervention. 
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After recording is complete, the subscriber can preview the generated video message 
(step 650). In one implementation, the subscriber uses a display toolbar 1017 to play, 
rewind, fast-forward, and/or otherwise view the recorded video message. If desired, the 
subscriber can delete the recorded video message by clicking the clear button 1035 and 
5 record a new video message. 

Referring to Fig. IOC, after determining that a recorded video message is acceptable, 
for example, by previewing (step 650), the subscriber and/or sender 602a can send the video 
message (step 652). In one implementation, the subscriber clicks the Send button 1030, 
which initiates the transfer of the video message to the recipient 602b. The transfer indicator 
10 1020 may change colors to indicate that video message data is being transmitted. 

Referring to Fig 10D, in one implementation, the Instant Video Message UI 1000 
includes only the send video message window 1015, the transfer indicator 1020, the Record 
C3 button 1025, the Send button 1030, the Clear button 1035, the receive move message window 
.7- 1040, and the transfer indicator 1045 to save screen real estate. This implementation may be 
f T5 especially attractive to a subscriber who does not like to type. 

ss:.:s 

\| The host 604 receives the video message from the client 602a (step 654). In one 

;Jf implementation, a server (e.g., IM server 5902, Domain Server 5904) on the host 604 is 
^ configured to receive video messages. The host 604 (e.g., IM server 5902, Domain Server 

5904) then authenticates the video message from the sender 602a (step 656). The host 604 
||f) may authenticate the video message, for example, by using a reverse look-up table to match 
Q the screen names and/or IP addresses associated with the sender 602a and/or recipient 602b 
to those of valid subscribers. In the event that either the sender 602a or recipient 602b is not 
associated with a valid subscriber, the host 604 reports an error message. 

Next, the host 604 moderates the video message session (step 658). In one 
25 implementation, a server (e.g., IM server 5902, Domain Server 5904) on the host 604 is 

configured to moderate a video message session between at least one sender 602a and at least 
one recipient 602b. Moderating may include managing load conditions of the host 604 by 
compressing, decompressing, caching, and/or allocating resources to efficiently store and 
forward video messages. Moderating also may include sampling and filtering video 
30 messages based on the capabilities and/or preferences of a recipient 602b. For example, the 
host 604 may sample a video message to determine viewing requirements and/or content of 
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the video message. If the host 604 determines that the intended recipient 602b is not capable 
of or does not wish to view a particular video message, the host 604 may discard, modify, 
and/or otherwise filter the video message. 

Moderating the video message session may include controlling the ability of at least 
one recipient 602b to view a video message. For example, the host 604 may structure the 
video message session such that one or more recipients view the video message 
simultaneously. This type of control may be useful in a lecture setting in which one sender 
602a is responsible for a majority of the video messages. Moderating also may include 
controlling the ability of at least one sender 602a to transmit a video message. For example, 
when several senders are participating in a video message, the host 604 may limit the ability 
to transmit a video message only to one sender at a time. This type of control may be useful 
in a conference setting in which many senders are transmitting video messages. 

Moderating the video message session also may include logging and/or storing video 
messages in a queue as they are received. If video messages are received from several 
senders, the host 604 may queue the video messages according to arrival, according to 
sender, according to topic, according to relevance, and/or according to any other ranking 
criteria. Video messages in the queue may be ordered, deleted, edited, and/or otherwise 
managed by the host 604. 

The host 604 also may allow the video message in the queue to be managed by at 
least one sender 602a and/or by at least one recipient 602b. In the implementation shown in 
Fig. 1 1, the recipient 602b displays a Instant Video Message UI 1 100 including a video 
message session window 1 150, a Start Saving Session button 1 152, a storage capacity bar 
1 154, a Play button 1 156, and a Delete button 1158. When clicked, the Start Saving Session 
button 1 152 requests the 602b and/or the host 604 to save the video message session (e.g., 
pending and viewed video messages). A video message session may be saved locally at the 
recipient 602b or remotely at the host 604 (e.g., in a temporary file). The storage capacity 
bar 1 154 indicates the percentage of storage capacity used. When the Play button 1 156 is 
clicked, a pending video message can be played out of order or a viewed video message can 
be replayed. When the Delete button 1 158 is clicked, a pending and/or viewed video 
message can be deleted. 
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In another implementation, the creator of a video message may decide that a pending 
video message has become irrelevant and instruct the host 604 to delete the video message 
from the queue. The host 604 would remove the pending video message from the queue and 
the pending video message would disappear from the video message session window 1150 
5 displayed by the recipient 602b. 

The host 604 then sends the video message to the recipient 602b (step 660). The host 
604 may send the video message to the recipient 602b without user intervention from the 
recipient 602b and/or may hold at the video message until receipt of a request from the 
recipient 602b. For example, referring again to Fig. 11, the host 604 may hold the video 
10 message and send a hypertext message 1007 identifying a pending video message to the 
sender 602b. The hypertext message 1 107 is displayed in the IM window 1 105, and when 
clicked requests the host 604 to download the pending video message to the recipient 602b. 
C3 When the video message is transferred from the 604 to the recipient 602b, the transfer 

if! 

^ indicator 1 145 may change colors. 

f f 5 The recipient 602b receives (step 662) and displays (step 664) the video message. In 

H| one implementation, the video message is displayed in the receive video message window 
3 1 140. The video message may be displayed automatically after the hypertext message 1 107 
is clicked or delayed until the display toolbar 1 142 is clicked. If the hypertext message 1 107 
KJ is clicked again after the video message has been viewed, the video message will be retrieved 
J & and replayed. The video message may be stored by the recipient 602b as a video file (e.g. 
O QuickTime file or AVI file). Stored video files may be replayed or transferred to other users 
as an e-mail attachment, for example. 

In one implementation, an active video messaging session uses three communication 
channels: a Generic Signaling Interface (GSI) channel, a Control channel, and a Video 
25 channel. The GSI channel is used by the video session to establish the initial connection. 
During this connection, the local IP addresses are exchanged. After the initial connection 
phase is done, the GSI channel is no longer used. By using the GSI channel, the exchange of 
local IP addresses is only done when both users authorize such an exchange by, for example, 
clicking on the Connect button 918. These actions protect users from having their local IP 
30 address automatically obtained without their consent. 


-23- 


Docket No.: 06975-133001 


The Control channel is a TCP/IP socket, where the IP address and port number of the 
remote side are obtained through the GSI channel. The Control channel is used to 
send/receive control attributes of the video session while the session is active. For example, 
because some firewalls will not allow an external connection to a socket on the inside of the 
firewall, the video tool attempts a connection from both sides of the session. This action 
allows a connection to be made if there is a maximum of one firewall within the connection. 
If there is a firewall on both sides, the chances are that no connection can be made and the 
video session will fail. To work across two firewalls, the user must obtain the port range 
used by video such that one of the firewalls can be modified to pass the range through. 

The Video channel is a TCP/IP socket used to transport video packets. This channel 
can either be UDP or TCP. In general, UDP is used since it minimizes latency. However, 
because some firewalls will not allow UDP packets to pass through, the video channel may 
have to use TCP. The video tool indicates the mode (i.e., TCP, UDP), or an auto mode 
where it attempts a UDP test, and upon failure resorts to TCP. 

Referring to Figs. 12 and 13, a video tool displays a UI 1200 and a UI 1300 for 
allowing a user to tailor video message functionality. The UI 1200 enables the user to 
control video message compression. The video tool also may include security features to 
protect the integrity of transferred data. For example, the video tool may compress data 
using a proprietary algorithm or may send the data in a proprietary protocol. The UI 1200 
also enables the user to control the volume, gain, and level for the speaker and microphone. 

The UI 1300 enables the user to control the display of the video picture. For 
example, the UI 1300 includes settings for white balance, gamma correction, exposure, 
flicker, flip, and/or other video features. 

The video tool may be any type of client controller (e.g., software, application, 
program) loaded on to a client device. The video tool responds to user interfaces and 
translates user commands into the appropriate actions with the client device. For example, 
the video tool opens, reads, writes, and closes the physical components on the client devices 
needed for video. The video tool also controls video and control channels with callbacks 
being executed to indicate status change. 

A number of implementations have been described. Nevertheless, it will be 
understood that various modifications may be made. In particular, a video-enabled IM client 
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can conduct messaging sessions with any type of IM client capable of messaging. For 
example, if the recipient 602b does not have the capability to receive a video message, the 
sender 602a may store the video message as a video file and then transfer the video file to the 
recipient 602b. The video file may be transferred as an e-mail attachment or transferred 
directly through a socket connection as described in commonly owned U.S. Patent 
Application No. 09/572,952, which is incorporated by reference in its entirety. In this way, a 
subscriber without camera equipment can still communicate with video-enabled IM clients 
by viewing a transferred video file and then responding to the video file by sending an instant 
message. 

Other embodiments are within the scope of the following claims. 
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